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ABSTRACT 

If the universe was reionized by O and B stars in an early population of 
galaxies, the associated supernovae should have enriched the universe to a mean 
metallicity Z = 10~^(1 + n^ec), where 

Tirec is the mean number of times that 
each baryon recombined during the reionization era. This is consistent with 
recent observations of the metallicity in the Lya forest at z ~ 3. The mean 
surface brightness observable at present from the galaxies that produced these 
heavy elements, in the rest-frame wavelengths 1216A < A ^ 2500A, should be 
~ 10~'^(Z/10-'')(Ofe/iV0.02) photons cm'^ g-i arcsec'^. Most of this radiation should 
be emitted at z > 5, before reionization was complete. 

These high-redshift galaxies may be detectable in near-infrared photometric 
surveys, identifying them via the Gunn-Peterson trough (analogous to the use of the 
Lyman limit cutoff to search for galaxies at z ~ 3, where the Lya forest blanketing 
is smaller). Their spectrum may also be characterized by a strong Lya emission line. 
However, the spectra of galaxies seen behind intervening gas that is still neutral should 
show the red damping wing of the Gunn-Peterson trough, with a predictable profile 
that obstructs part of the Lya emission. 

The low-mass galaxies formed before reionization might constitute a distinctive 
population; we discuss the signature that this population could have in the faint 
number counts. Although most of these galaxies should have merged into larger ones, 
those that survived to the present could be dwarf spheroidals. 



Subject headings: galaxies: formation - large-scale structure of universe - quasars: 
absorption lines 



1. Introduction 



Observations of large-scale structure have, over the last few decades, strongly supported 
hierarchical models of gravitational instability with "cold dark matter" . Density fluctuations grow 
to non-linearity first on small scales. High-density halos are formed, which then continue to accrete 
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low density matter around them and merge with each other to form objects on progressively 
larger scales. Evidence for this general picture includes the observed hierarchical nature of galaxy 
clustering and the galaxy peculiar velocity field (e.g., Strauss & Willick 1995), the presence of 
substructure in galaxy clusters showing that clusters are being assembled at the present time 
from smaller units (e.g.. Bird 1994), the Lya forest at high redshift showing that structures 
on smaller scales than the present clusters were collapsing in the past (Ranch et al. 1997 and 
references therein), and the CMB fluctuations (White, Scott, k, Silk 1994). Even though the origin 
of the initial fluctuations and the nature of the dark matter remain in the realm of speculation, 
and the exact form of the linear power spectrum of density fluctuations is uncertain, a generic 
consequence of this scenario is that the first galaxies to form in the universe had very small scales 
and masses. The earliest galaxies should have formed in the first collapsed objects where the 
baryonic component was able to lose its energy through radiative cooling, concentrating to the 
center of dark matter halos and becoming self-gravitating, leading to the formation of the first 
stars. In the absence of cooling, the gas virialiscs and remains in hydrostatic equilibrium in dark 
matter halos and galaxies cannot form (e.g.. White k. Rees 1978 and references therein). 

As more massive structures form, the velocity dispersion of collapsed halos increases, as 
does the temperature of the gas in these halos. The gas that forms the first galaxies is, of 
course, initially neutral. For T 5 x 10^ K, the only effective coolant in gas with the primordial 
composition of hydrogen and helium is molecular hydrogen (e.g., Tegmark et al. 1996). Molecular 
cooling almost certainly played a role in forming the very first stars, but the fraction of molecules 
is always small and they should be rapidly photodissociated by UV photons from the stars 
themselves (Haiman, Rees, &: Loeb 1997; Ostriker & Gnedin 1997). Radiative cooling is then 
suppressed until temperatures ~ 10^ K are reached, when atomic hydrogen provides efficient 
cooling through line excitation and collisional ionization by electrons in the Maxwellian tail of the 
energy distribution. The cooling rate rises steeply with temperature, so the gas in this regime 
contracts almost isothermally. 

The first-generation galaxies will eventually provide enough UV photons to ionize all the 
diffuse gas in the universe (e.g., Couchman k. Rees 1986); active nuclei could also start forming 
and contribute to the process of "reionization" (e.g., Arons & Wingcrt 1972). The total amount 
of star formation that precedes reionization obviously depends on the IMF; it also depends on 
what fraction of the Lyman continuum from O and B stars escapes into the intergalactic medium, 
rather than being absorbed by gas in the star-forming regions themselves. After reionization has 
occurred, there is a universal UV background high enough to maintain the HI fraction of any 
diffuse gas at a very low level; this substantially reduces the cooling rate at temperatures below 
~ 5 X 10^ K. This ionized diffuse gas can form the structures responsible for the observed Lya 
forest (Cen et al. 1994, Miralda-Escudc et al. 1996, hereafter MCOR, and references therein). The 
higher entropy of the ionized gas, together with the suppressed cooling rate, inhibit its infall into 
halos with virial temperatures much below 10^ K (e.g., Efstathiou 1992; Thoul & Weinberg 1996); 
dissipation is only efficient in more massive objects, and presumably leads to the formation of 
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most galaxies that we have observed at ah redshifts back to z ^ 5. 

The physics of the galaxy formation process is probably substantially different depending on 
whether they form from neutral or from photoionized gas, and this suggests that galaxies formed 
in two distinct phases (e.g., Haiman & Loeb 1997a). In this paper, we shall designate "Population 
A" those galaxies formed from accretion of photoionized gas, and "Population B" those galaxies 
formed in the absence of an external ionizing background, so that photoionization from external 
sources was negligible during the dissipation process. Essentially, the Population B formed before 
the reionization epoch and emitted the photons that reionized the universe, and the Population A 
formed afterwards, although in practice reionization is a gradual process where different regions 
will be photoionized at different times, as they are immersed in expanding cosmological HII regions 
that eventually overlap. 

The distinction between these two populations is so far only a theoretical concept, and our 
understanding of the effects of reionization on galaxy formation is likely to remain limited until 
the existence of the Population B (hereafter, Pop B) can be observationally tested. In this paper 
we discuss the constraints that we can put on the characteristics of the Pop B galaxies given 
our present knowledge of the high redshift universe, and the prospects for their detection. In 
a previous paper (Miralda-Escude k. Rees 1997, hereafter MR97), the possibility of detecting 
individual supernovae from these galaxies was investigated, pointing out that although supernovas 
at z ^ 5 should be extremely faint, their host galaxies could be even fainter, given the small masses 
expected for the Pop B galaxies. But as we shall see here, the luminosities of Pop B galaxies may 
in fact be not much lower than that of a supernova. Moreover, their detection is facilitated because 
they should be exceedingly numerous, and because of the distinct spectral signature expected from 
starburst galaxies at high redshift: the Gunn-Peterson trough and a possibly strong Lya emission 
line. 

2. The Brightness of Stellar Radiation Emitted at Reionization 

It has recently been shown that the gas in the Lya forest was already enriched to a heavy 
element abundance Z ~ by a redshift z = 3, from observations of absorption lines of 

CIV and other species associated with Lya forest lines with 10^^'^ cm~^ < Nhj ^ 10^^ cm~^ 
(Songaila & Cowie 1996 and references therein). Detailed calculations of the expected column 
densities of the observed absorption lines, using hydrodynamic simulations of the Lya forest and 
realistic models for the spectrum of the ionizing background (Ranch, Haehnelt, &: Steinmetz 1996; 
Hellsten et al. 1997) have shown that the carbon abundance needed to reproduce the observations 
is [C/H] = —2.5. The metal abundances arc similar to those of Population II stars, where oxygen 
is the most abundant element and is overabundant by a factor ~ 2 relative to carbon. With 
Zq = 0.02, the metallicity of the Lya forest is then Z ~ 10~^. As argued in MR97 this metallicity 
should be approximately the same as the mean metallicity of the universe at 2; = 3 if the Lya 
forest contains most of the baryons (as is found to be the case in models of structure formation 
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for the Lya forest similar to those analyzed in Hernquist et al. 1996 and MCOR), and if the Lya 

forest metallicity is uniform. Actually, the metallicity probably increases with Nhi'- Hellsten et 
al. (1996) find that the absorbers with Nm < 10^^ cm~^, which should contain a large fraction of 
the baryons, should have lower metallicity than [C/H] = —2.5, the value they need to account for 
the CIV column densities of absorbers with higher Nhi- The increase of metallicity with Nhi is 
also predicted in models of the enrichment of the IGM (Gnedin &; Ostriker 1997). This can result 
in a lower mean metallicity. In fact, Songaila (1997) has shown that by adding the column density 
of the CIV lines, the lower limit to the mass of metals that is obtained is smaller by a factor ~ 10 
compared to the value we assume here. Thus, significant uncertainties still remain on the mean 
metallicity. 

The ratio of the mass of heavy elements ejected by a star to the energy in ionizing photons 
emitted over the lifetime of the star, as derived from models of stellar evolution and supernova 
explosions, turns out to be about constant over the relevant mass range lOM© ^ M ^ 50Mq 
according to stellar evolution models (although the possibility that some massive stars collapse 
to black holes without ejecting heavy elements introduces some uncertainty), so given a mean 
metallicity Z we can predict the energy in ionizing photons that was emitted for each baryon in 
the universe. According to Madau & ShuU (1996), this energy is 0.002ZmpC^ per baryon. As 
discussed in MR97, for Z = 10~^ we find that 10 ionizing photons were emitted per baryon by the 
stars that produced the Lya forest heavy elements (assuming a mean energy of 20 eV per ionizing 
photon). This is probably the radiation produced by the Pop B galaxies, although it is certainly 
possible that after the Tinivcrsc was reionized, some of the heavy elements produced in the newly 
forming, more massive Pop A galaxies were also dispersed to the intergalactic medium (hereafter, 
IGM) and contributed to the Lya forest metal abundance. 

One of these 10 ionizing photons was used to reionize the universe. The majority of the 
other photons must also have been absorbed by neutral hydrogen. A fraction fi of the photons 
will be absorbed internally, in the galaxies where the emitting stars were formed; the rest 
should be absorbed after having escaped their original galaxies, in other dense absorbing systems 
where protons can recombine many times during the reionization epoch (similar to the observed 
Lyman limit systems). Of the photons that were absorbed internally, about 70% resulted in the 
production of a Lya photon, and the other 30% resulted in emission in the two-photon continuum 
from the 2s state (see Table 9.1 in Spitzer 1978; we assume that the internal absorption takes 
place in regions that are sufficiently optically thick to absorb essentially all ionizing photons 
produced by direct recombination to the ground state). The Lya photons will then be scattered 
many times in the host galaxy, until they move to the wings of the Lya line and escape. Some of 
them may be absorbed by dust before escaping, so we define as the fraction of Lya photons 
that can eventually escape without being absorbed by dust. The mean comoving number density 
of these Lya photons is then 7 /i/d(Z_4)n5, where iih is the comoving baryon density and 
Z-4 = Z The mean surface brightness of the Lya emission line from the Pop B galaxies 
is simply obtained by multiplying the photon comoving number density by c/47r, which yields 
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10 ''/i/d^_4(fib^^/0.02) photons cm ^ arcsec ^). The fiducial value of Jlfo/i^ used here is close 
to that implied by a primordial deuterium abundance D/H 3 x 10~^, as reported by Tytler, 
Fan, & Buries (1996). 

This determines the average surface brightness we receive in the sky from these Pop B 
galaxies in the Lyct emission line, or in other words, the product of the number of Pop B galaxies 
per solid angle times their individual, mean Lya flux. A much more uncertain question is the 
flux from each individual galaxy; the uncertainty is essentially tied to the efficiency of forming 
massive stars in collapsed dark matter halos as a function of their velocity dispersion during the 
reionization epoch. To see the range of possibilities we might have for the main characteristics of 
Pop B galaxies, we shall examine three possible scenarios: 

(a) Star formation is highly efficient in all collapsed halos with velocity dispersion 
a ^ 15Kms~^, corresponding to the virial temperature T = fia^/k ~ 10^ K where rapid cooling 
by atomic hydrogen can start taking place. In this case, reionization would take place as soon as 
the highest density peaks on the scales of these low velocity dispersions collapse. As a specific 
example, let us take the top-hat spherical collapse model for a halo, where a spherical region of 
mass M turns around at a radius Rt when the age of the universe is t//2, and collapses at time tf 
(e.g., Peebles 1980, §19), forming an isothermal halo with = GM/Rt (obtained assuming that a 
mass M is contained within a radius in the final equilibrium configuration). The halo mass 

is then M = 2^/'^ / {2T:G)a^tf ~ lO9M0(o-/2OKms-^)''^ {tf /l^fyr). If a fraction h of the mass 
turns to stars (with close to for the high efficiency case) over a timescale ts ~ 0.3tj of order 
the free-fall time of the halo, the star formation rate is O.3M0 yr~^(c7/20Kms~^)^ (/b/0.1). For a 
normal IMF, this can yield a luminosity close to the peak luminosity of a supernova. The implied 
metal production rate depends of course on the IMF, but is ~ 10~^ times the star formation rate 
when we calibrate it from the observed yields in normal galaxies where the average metallicity is ~ 
1% after a large fraction of the gas has turned to stars. Given the numbers discussed above for the 
luminosity in Lya photons, which imply that 10^ fi fd Lya photons are emitted for each baryon 
that is expelled from stars in the form of heavy elements, the Lya luminosity from this efficient, 
rapid starburst in a Pop B galaxy would be lO^Vi /d ((^/20Kms-^)3 (/b/O.l) photons s'^. 
The corresponding Lya flux at present (for $7 = 1 and Hq = 50Kms~^ Mpc~^) is 
5x 10-^/i/rf((j/20Kms-i)3(/fe/0.1) [l - (1 + z/)-V2l (1 + z/)-i photons cm-^ s'^ If 
the sources that reionized the universe have these characteristics, then we can use the mean Lya 
surface brightness we obtained earlier to conclude that the number density of these sources in the 
sky should be ~ 1 per arcsec^ for the fiducial numbers we have chosen, and a formation redshift 
1 + ~ 10. The redshift Zf should of course depend on the detailed model for the amplitude of 
the primordial fluctuations on small scales. 

The main problem with this scenario is that, as pointed out by several authors (e.g., Dekel 
Sz Silk 1986), the energy from supernova explosions in galaxies of such low velocity dispersions 
should easily be able to expel the gas from the halo when only a small fraction of the gas has 
turned to stars. In fact, the enriched gas needs to be expelled at some stage to spread the heavy 
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elements through the IGM. Although one could imagine the supernovae taking place in a very 
dense medium where the energy was quickly radiated as soft X-rays, it seems likely that the 
supernova explosions will reduce the efficiency of star formation by a large factor, due to the 
heating and expulsion of gas. Indeed, it has been argued that this reduction of the star formation 
rate is necessary to prevent too many baryons from turning to stars in small galaxies at early 
times (e.g., Navarro & Steinmetz 1997). This leaves us with two other types of Pop B galaxies 
that could be the sources of the reionization photons, our scenarios (b) and (c). 

(b) Reionization is caused mostly by low- mass objects similar to case (a), but only a small 
fraction of the gas in each object turns to stars before the gas is expelled in a wind. Thus, the 
parameter ft, <C fife, so each Pop B galaxy emits fewer photons and a larger number of them have 
to form, which therefore have to originate from lower amplitude peaks. The luminosity of each 
galaxy would be reduced proportionally to due to the smaller mass of stars. This might be 
partly compensated by having starbursts of very short duration (in principle, starbursts could 
occur over a time much shorter than the free-fall time through the dark matter halo if only a small 
fraction of the halo gas turns to stars). 

(c) The efhciency of massive star formation in the first halos where atomic cooling takes 
place is so low that not enough photons are emitted to complete the reionization until more 
massive galaxies (with a ^ 100 Km s""*^ collapse and form stars more efficiently. In this case, the 
Pop B galaxies responsible for the reionization would have relatively large masses, well above the 
Jeans mass for photoionized gas, so reionization might be of little consequence for the process of 
galaxy formation, and the Pop B galaxies might not have any observational characteristic that 
would distinguish them from Pop A galaxies (this is because photoionization has little effect on 
the cooling rate in halos with velocity dispersion a ^ lOOKms"^; see Thoul & Weinberg 1996, 
Navarro & Steinmetz 1997). 

The inefficiency of star formation in low a galaxies might arise from two different reasons. 
One is the rapid expulsion of the gas after a very small number of stars have formed, as in case (b). 
The other possibility is that the gas is not expelled at all, but instead forms rotationally supported 
disks by radiative cooling and dissipation, and then the gas in these disks stays in atomic form 
and does not effectively turn to stars. This could happen because the low-metallicity gas might 
not cool to the cold phase of the interstellar medium (see Corbelli &; Salpeter 1995). Staying 
at a temperature T ~ 5000 K in a disk with a low circular rotation velocity, the disks might 
be gravitationally stable according to Toomre's criterion, preventing the formation of molecular 
clouds. These stable disks might persist until their host halos merge within larger systems, when 
the disks would be disrupted and the gas would again dissipate and form a new, more massive disk, 
with a higher efficiency of star formation. A low star formation efficiency in low a galaxies could 
also be part of the solution to the common problem that hierarchical models have in producing 
too many low-mass galaxies (e.g., Kauffmann, Guiderdoni, & White 1994). 

We need to point out here that moderately massive galaxies are likely to form during 
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reionization, even in cases (a) and (b), because the power spectrum at small scales in cold dark 
matter models generically flattens to a slope close to n = —3, so the amplitude of fluctuations 
decreases very slowly with scale. As an example, let us consider the standard cold dark matter 
model with h = 0.5, and the rms density fluctuation normalized at present to as = 0.7 on spheres 
of Sh"^ Mpc. This model has been shown to be approximately in agreement with the observed 
characteristics of the Lya forest (Hernquist et al. 1996, Dave et al. 1997, Ranch et al. 1997), and 
therefore should probably have about the right amplitude of density fluctuations on small scales. 
The three solid lines in Figure 1 show the velocity dispersion (or virial temperature Tyir = ^a'^/k) 
of halos having collapsed from (l,2,3)-(7 fluctuations as a function of rcdshift (assuming 5c = 1.69 
for the critical value of the linear overdensity that corresponds to the formation of a galaxy, as 
in the spherical collapse model). The dashed lines indicate constant halo mass. If, for example, 
reionization was complete at z ~ 9, when halos with a = 20Kms~^ would just be collapsing from 
2 — a peaks, at the same epoch wc would have halos with a = 75Kms~^ (with a mass ~ 50 
times larger than halos with a = 20Kms^^) collapsing from 3 — o" peaks on this larger scale. In a 
Gaussian theory, the 3 — cr peaks should contain 10% as much mass as the 2 — a peaks at a fixed 
epoch, so we see that the mass distribution of the Pop B galaxies should probably extend well 
above the minimum mass for efHcient atomic cooling. Notice that the very large range in mass of 
collapsed objects at high redshift is a generic consequence of the flattening of the power spectrum 
on small scales, due to the fact that the universe was radiation dominated when these small scales 
entered the horizon. Objects from 1-a peaks, which contain most of the mass, probably never 
cooled before reionization because they were ionized by the radiation from more massive objects, 
and could only start collapsing at z ~ 3, at a ~ 30Kms^^, when the cooling rate was again fast 
enough. But objects from 3-a peaks at z = 9 could already have total masses over lO^^M©. 



3. Detecting Galaxies at z > 5 

We now discuss the prospects for detection of this population of galaxies at high redshift. 
First, we notice that given the mean surface brightness of Lya photons derived previously, the 
surface brightness in the UV continuum also follows with relative little additional uncertainty, 
because the stars emitting most of the light in the range 1216A < A ^ 2500 Aare also mostly very 
young stars. Given the typical UV spectrum of a starburst (e.g., Fig. 1 in Bruzual 1983), we infer 
that for every Lya photon produced from ionizing photons there should be ~ 10 UV continuum 
photons (because the energy emitted in ionizing photons is ~ 1/3 of the UV energy emitted to the 
red of Lya in a young starburst, and 1/3 of the energy in ionizing photons is converted to Lya 
photons). This implies that the mean surface brightness from all Pop B galaxies in the rest-frame 
UV continuum is 10~^ photons cm~^ arcsec"^. The value of this mean surface brightness agrees 
with the calculation by Haiman &; Loeb (1997b), who used also the metallicity of the Lya forest 
as a constraint and made a more detailed model of the stellar population and the distribution of 
galaxy luminosities. 
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The main spectral feature that should identify any such galaxies at z ;^ 5 is a sharp break of 
the UV continuum at the Lya wavelength, due to the Gunn-Peterson trough (Gunn & Peterson 
1965); in addition, the Lya emission line may be present, depending on internal dust absorption 
and scattering of the Lya photons (see the next section). Notice that the redshift at which the 
IGM was reionized is not highly relevant regarding the presence of the Gunn-Peterson trough, 
because even if the medium was reionized at 2; S> 5, we know that the flux decrement caused by 
the Lya forest reaches a factor of 2 at 2: ~ 4 and grows rapidly with redshift. Thus, the technique 
of identifying galaxies at z ~ 3 from the Lyman continuum break (Guhathakurta et al. 1990; 
Steidel ct al. 1996) should be replaced by the Gunn-Peterson trough at z ^ 5 (see Madau 1995 
and Madau et al. 1996 for a careful analysis of the effects of the Lya forest on galaxy colors), 
although some residual flux between the Lya line and the Lyman limit would still be present up 
to 2; ~ 7 if the reionization occurred earlier, and the Lyman limit discontinuity may then be used 
in conjunction (see Loeb Sz Haiman 1997). 

The mean surface brightness in the rest-frame UV continuum from the Pop B galaxies can 
also be expressed as Siy = 6 x 10~^^ ergcm"^ s~^ Hz~^ arcsec"^ = 32ABarcsec~^ (where AB 
denotes AB magnitudes in the band where the UV continuum is observed; notice that in these 
units, the derived surface brightness is also independent of redshift because the (1 + z) factor due 
to the redshift of the photons is cancelled since we express the surface brightness in energy per 
unit frequency). Here we have adopted the values ilb/i^ = 0.02, Z = 10~^; the surface brightness 
is of course proportional to these two quantities. If we now take the example of case (a) in the 
last section, where there is one Pop B galaxy per arcscc^ at redshifts Zf ~ 10 with a c± 20Kms~^ 
(corresponding to 2-a peaks in Fig. 1), each such galaxy would have an AB magnitude of 32. 
However, as mentioned earlier, realistically these high-z galaxies should have a wide range of 
luminosities, corresponding to the range of their masses. The dotted lines in Figure 1 indicate 
the AB magnitudes of galaxies in halos during a starburst, under the same assumption as in §2 
that all the baryons turn to stars over the time tg ^ OMf. The AB magnitude is for rest- frame 
wavelengths in the UV continuum, longward of Lya . Thus, galaxies forming from 3 — a peaks 
could have AB magnitudes of 28 even at z 10. Even though these would be ~ 500 times more 
rare than 2 — a peaks at the same redshift, the number density could still be close to one galaxy 
per square arc minute. Our scenarios (b) or (c) would be more optimistic because, for a fixed 
epoch of reionization, a model with a higher amplitude of density fluctuations and a greater 
number of massive halos at high redshift would be required due to the lower overall efficiencies of 
star formation and production of ionizing photons. 

These high-redshift galaxies will probably be detectable soon. Detection with ground-based 

telescopes is hindered by the high sky background in the infrared, but may still work at z ^ 6, and 
in Lya searches where special wavelengths with low atmospheric emission are chosen. The Keck 
telescope can detect point sources to AB magnitudes R < 28, and / < 27, in a night of observing 
time (M. Ranch 1997, priv. communication; Cohen 1995a,b). The magnitude limit could be 
improved with adaptive optics. With space instruments, the faintest galaxies in the HDF reach 
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to / ~ 28.5 (Williams et al. 1996). The New Generation Space Telescope could image galaxies to 

AB magnitudes ~ 31 in the near-infrared, and should be able to detect galaxies to much higher 
redshift (Mather & Stockman 1996), with a much higher number density tlian has been seen so 
far. The prospect for detecting the high-redshift galaxies responsible for the enrichment of the Lya 
forest has also been analyzed in Haiman &; Loeb (1997b), Loeb & Haiman (1997), Loeb (1997) 
and Cen (1997). 

In general, the lensing magnification in rich lensing clusters may be used here to stretch 
the magnitude limit (see also Cen 1997). As an example, a lensing cluster with an Einstein 
ring radius h = 30" should magnify to A > 10 an area of ~ 30 arcsec^ in the source plane. In 
the example used above, about 30 galaxies with AB = 32 could be in this area, which would 
be magnified to AB = 29.5. Magnified images of high-redshift galaxies should characteristically 
appear in pairs around the critical lines, in a region that can be predicted from lensing models 
(see Miralda-Escude &; Fort 1993; Kneib et al. 1996, and references therein), so this should help 
in their identification. These numbers indicate that a new deep field (similar to the HDF) imaged 
with HST in a rich cluster, adding also the H and J filters in the near-infrared, might well identify 
several galaxies at z > 5. In fact, the largest redshift object known at present (at z = 4.92) is 
already a gravitationally lensed galaxy (Franx et al. 1997). 

We have assumed in this discussion that these high-redshift galaxies would be sufficiently 
small to remain unresolved. Resolved object would need to have higher fluxes to be detected, 
since the detection is limited by the sky background. The scale-lengths of the Pop B galaxies are 
expected to be very small. The radius of a collapsed halo scales as atf. Compared to present 
day galaxies, the Pop B galaxies at z > 5 should have a formation time smaller by at least a 
factor ~ 10, and a velocity dispersion that is also smaller by a factor ^ 5. We can reasonably 
assume that the ratio of the size of the region where stars form to the radius of the collapsed dark 
matter halo is similar for all types of galaxies, if the processes of dissipation and distribution of 
the angular momentum that determine the size of rotating disks (Fall &: Efstathiou 1980), or the 
onset of star formation in spheroidal components when a gas cloud becomes self-gravitating and 
fragmentation occurs, are similar in galaxies of different masses. Therefore, given the typical scales 
of a few Kpc for present-day galaxies, the scales of Pop B galaxies are likely to be smaller than 100 
pc, corresponding to angular sizes ~ 0.01 arcsec. This would remain unresolved even with NGST. 

4. The Lyman Alpha Emission Line 

Another possible way to detect the faint Pop B galaxies is to search directly for the Lya 
emission line. As we discussed before, for a normal starburst spectrum we expect 10% of the 
UV photons to be in the Lya emission line if dust absorption is not important. Therefore, if the 
sensitivity for detecting galaxies is still limited by the sky background for emission-line searches, 
the width of the line should be AA/A < 0.01 to allow detection on a shorter time than for the UV 
continuum. The width of the Lya line in emission from a region of neutral gas with column density 
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Nhi = 1022iV22cm-2, and velocity dispersion a = lOcJeKms"^ is AA/A ~ 2 x 10"^(A^22O-6)^''^ 
(Harrington 1973; this formula is valid for a very optically thick system where photons are 
scattered many times in the damping wing before escaping from the system). Thus, the sensitivity 
of emission-line searches might be at best similar to searches for the UV continuum, if the sky 
background were smooth in wavelength. 

From the ground, line-emission searches have the advantage that a wavelength of particularly 
low atmospheric emission can be selected, so the line-emission search can then be substantially 
more sensitive than a photometric one. The deepest Lya emission surveys so far have reached 
limits of 3 X lO^^photons cm~^ s~^ at z ~ 4 (Thompson, Djorgovski, & Trauger 1995), and 
10~^photons cm^^ s^^ at z ~ 7 (Parkes, Collins, & Joseph 1994). With our assumed ratio of 10 
UV continuum photons for each Lya photon (and ignoring the possibility of dust obscuration), 
these limits correspond to AB magnitudes of 28 and 24.5 in Figure 1, respectively. At z ~ 4, the 
population of luminous starbursts must be obscured by dust, as shown by the recent discovery by 
Steidel et al. (1996) of the Lyman limit dropout galaxies with AB ~ 26 at this redshift. But the 
lower mass galaxies at higher redshift might have stronger Lya emission lines. Some examples of 
high redshift galaxies with strong Lya emission have been found (Hu & McMahon 1996; Franx et 



The Lya emission line may be suppressed due to internal dust extinction. In addition, any 
photon emitted in the blue side of the Lya line will be scattered in the surrounding IGM, reducing 
the strength of the line by a factor of 2. For a galaxy observed when the surrounding IGM is 
still mostly neutral, even photons on the red side can be scattered due to the damping wing of 
the Gunn-Peterson trough. Assuming a neutral IGM with uniform density, the Gunn-Peterson 
optical depth is tgp = 2.1 x 10^ [f^b/i(l — y)/0.03] [(1 + z)/6]^/^ (where Y is the primordial helium 
abundance). The optical depth at a wavelength A A to the red of the Lya line is 



where = A/(47rz^Q) = 2.02 x 10~^, A = 6.25 x lO'^s"-'^ is the decay constant for the Lya 
resonance, and Va = 2.5 x 10^^ Hz is the frequency of the Lya line. This expression can be 
approximated to 



r(AA) = TGpi?a/vr(AA/A)-^ = 1.3 x 10^^ [Qbh{l - y)/0.03][(l + z) {^\/ Xy^ . (2) 



This width of the damped absorption from the IGM is of similar breadth as the emission lines we 
may expect from starburst galaxies, so the red side of the Lya emission line should be partially 
suppressed by the IGM in a galaxy observed before the reionization. It is possible that the ionizing 
radiation of the galaxy itself would have ionized the surrounding IGM, but this is unlikely in a 
source with a strong Lya emission line because most of the ionizing photons need to be absorbed 
to produce the line. The edge of the Gunn-Peterson trough should have the shape given by 
equation (|2|), except for the fact that the inhomogeneity of the IGM may alter the profile (in 



al. 1997). 




(1) 
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particular, the presence of a halo of gas accreting on the galaxy may substantially increase the 
column density contributing to the damped profile from gas at a redshift close to that of the 
galaxy). This damping wing is studied in more detail in Miralda-Escude (1997). 

5. Effect of the Population B on the Faint Galaxy Number Counts 

We have calculated in this paper that the galaxies where the heavy elements observed 
in the Lya forest at z ~ 3 were synthesized have to contribute a mean surface brightness of 
32 AB arcsec"^ to the galaxy number counts. The majority of these galaxies should be at redshifts 
higher than any galaxy redshifts determined so far {z ^ 5), due to two reasons: first, it is 
unrealistic that the massive stars could have formed in a highly synchronous fashion, shortly 
before the redshift where the heavy elements are observed; and second, the dispersion of the heavy 
elements through the Lya forest should take a substantial fraction of a Hubble time, because the 
enriched gas is not likely to be expelled from the halos around galaxies at a velocity much higher 
than the escape speed. 

Let us now compare this predicted mean surface brightness for the Population B with the 
mean surface brightness of the faintest galaxies observed so far. We shall do this in the I and K 
bands. In the I band, the faintest galaxies in the HDF are at / ~ 28.5, reaching a number density 
at this magnitude of 5 x 10^ deg~^ mag~^ (Williams et al. 1996). The slope of the counts at this 
magnitude is ~ 0.2 (meaning that the counts arc increasing as 10°'^^). If we assume that this slope 
remains constant at all fainter magnitudes, the total mean surface brightness from galaxies with 
/ > 28.5 is 31.2 AB arcsec"^. Because this is only a factor 2 brighter than the expected value for 
the Pop B galaxies, we can conclude that if all the Pop B galaxies were at ^ < 6 (and therefore 
still observable in the I band), then either these Pop B galaxies should be the objects detected 
just at the magnitude limit in the HDF, or the slope of the I counts should rise again at fainter 
magnitudes to a value greater than 0.4, to yield a mean surface brightness comparable to the one 
contributed by the galaxies with / ~ 29. The first possibility is probably ruled out from the work 
of Madau et al. (1996), who find that only a small fraction of the faint galaxies in the HDF can be 
at z > 3.5 (see their Fig. 8a). 

Actually, the I counts do not need to steepen again at I > 29, as long as most of the surface 
brightness inferred for the Population B comes from galaxies with 2; > 6, since the Gunn-Peterson 
trough will then cause these galaxies to drop out of the I band. But the same argument can be 
applied at longer wavelengths as better observations become available in the J, H and K bands in 
the future, from HST and NGST. If the counts were observed to flatten to a shallow slope, and the 
mean surface brightness from the faintest observed galaxies were already less than 32 AB arcsec^^, 
while the redshifts of these galaxies were still not significantly higher than the redshift at which 
the heavy elements were present in the Lya forest, then we could infer that the slope of the galaxy 
counts should steepen again; in other words, there should be a "second hump" , a maximum in the 
contribution to the mean surface brighness as a function of galaxy magnitude, due to the Pop B 
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galaxies. The presence of a "second hump" would be evidence that the Populations A and B are 
really distinct, as in the cases (a) and (b) discussed in Section 2 (unless dust obscuration could 
cause the dip between the two "humps" in the galaxy counts). 

The present observations in the K band from the ground are not yet deep enough to put 
significant constraints along this line of argument. The faintest counts determined so far yield a 
density 2.2 x 10^ deg~^ mag~^ at K = 26.4 (Djorgovski et al. 1995; the AB magnitude K = 26.4 
corresponds to = 23.5 in the Johnson system). Assuming a constant slope of 0.3 for the faint 
counts (the observed slope in the range 23 < K < 26.4), the mean surface brightness due to 
galaxies with K > 26.4 should be 29.2 AB arcsec"^, still larger by a factor ~ 10 compared to the 
expected value for the Population B. Thus, the Pop B galaxies could appear in the K counts at 
K 3> 26.4 without requiring a steepening of the slope. 

6. Dwarf Spheroidal Galaxies: Remnants of the Population B? 

If the formation of galaxies occurred indeed in two distinct populations, due to the reionization 
of the IGM, we might then expect that any remnants of the Population B that survive in the 
universe at the present time would also constitute a special morphological class of galaxies. 

We propose here that the dwarf spheroidal galaxies may be such remnants. Dwarf spheroidals 
are galaxies with luminosities in the range 10^ — W^Lq, with old stellar populations and devoid 
of gas. The velocity dispersion of the stars is often near lOKms"^, and the surface brightness is 
usually low. The mass-to-light ratios are generally much higher than expected for an old stellar 
population, even in the center, and they vary over a wide range. Thus, the mass is dominated 
by dark matter at all radii, and the inferred central dark matter densities arc higher than in any 
other known galaxies (see Gallagher & Wyse 1994 for a review). These galaxies have only been 
studied in the Local Group and a few nearby clusters, owing to their low luminosity. 

These characteristics are similar to what should be expected in cases (a) or (b) discussed in 
Section 2. In this scenario, the dwarf spheroidals would have been formed in dark matter halos 
with cr ~ 10 — 50 Km during reionization. A large fraction of their gas could have been expelled 
in a wind, which would have caused the stars to expand adiabatically in the dark matter potential 
well as the central baryonic mass was lost. Most of these galaxies should then have merged into 
more massive systems (forming part of the spheroid population of stars in normal galaxies like the 
Milky Way; see MR97), but some could survive until today in orbit in galactic halos, or even have 
remained as isolated objects. Many dwarf spheroidals show evidence for multiple starbursts, some 
of them having taken place only a few billions years ago. This does not contradict our picture, but 
it implies that after the initial starburst at reionization, dwarf spheroidals were able to accrete 
more gas and undergo other starbursts at later times. This could happen when the accreted gas 
becomes self-shielding to ionizing radiation and cools rapidly, as in the scenario outlined by Babul 
&; Rees (1992). Accretion of new gas should occur before these dwarfs are captured in an orbit 
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within more massive galactic halos, because after this capture the surrounding gas will be shocked 
to high temperature and will move at a high velocity relative to the dwarf spheroidal. 

7. Conclusions 

Our present understanding of the evolution of the IGM, of its reionization and its metal 
enrichment, leads to the expectation that the first galaxies in the universe were formed at very 
high redshifts, while the IGM was not yet fully ionized. These galaxies might have very different 
properties from the galaxies we are familiar with in the present universe; they affected the physical 
state of the intergalactic gas that later formed the more massive galaxies like the Milky Way, 
and the stars that formed in them are probably part of the spheroidal components in present-day 
galaxies. The discovery of these high-redshift galaxies is a crucial and necessary step to advance 
to a more complete understanding of how galaxies formed. 

The requirement of having produced the density of heavy elements in the Lya forest leads 
to a prediction of the mean surface brightness contributed from the Population B galaxies. This 
mean surface brightness is not much smaller than that due to the faintest galaxies so far detected 
with the HST and ground-based telescopes. Detection of these first galaxies, for reasonable 
estimates of their luminosities, should be possible with the proposed NGST, which is an ideal 
tool for the detection of star-forming galaxies at 5 ^ z ^ 20 (Mather &; Stockman 1996), using 
the Gunn-Peterson trough to identify them. At present, the brightest examples of galaxies at 
these redshifts might already be found with HST and Keck, possibly with the help of lensing 
magnification in the fields behind rich clusters of galaxies. 

We thank Len Cowie, Richard Ellis, Abraham Loeb, Max Pettini, and Michael Ranch for 
useful discussions and for helping us with the numbers for the sensitivity of the Keck telescope. 
We thank Chung-Pei Ma, who suggested to us the nomenclature of Populations A and B, and 
Bohdan Paczynski and David Spergel, for discussions on the properties of dwarf spheroidals. 
We are grateful to David Weinberg for providing a code that was used for calculating the halo 
abundances for Figure 1. 
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Fig. 1. — Thick solid lines give the virial temperature and velocity dispersion of halos collapsing 
at redshift z, for (l,2,3)-fT peaks, in the cold dark matter model with ^1 = 1, h = 0.5, as = 0.7. 
The dashed lines are for constant halo mass. The high-cj peaks collapse on a larger scale than the 
low-cr peaks at a fixed epoch, hence they have higher velocity dispersion and masses. The dashed 
lines give the AB magnitude of a galaxy formed in a given halo, assuming that all baryons in the 
halo (accounting for 8% of the total mass) turn to stars in a starburst lasting for 0.3 times the age 
of the universe. The AB magnitude is at a rest-frame wavelength in the UV, longward of Lya . 



